Methods and Systems for Automatic Media Output Based on User Proximity

ABSTRACT

This application discloses a method implemented on a casting device to control display of media content on an output device coupled to the casting device. After it is determines that the output device operates at an inactive state, the casting device detects a client device associated with the casting device, and determines a proximity of the client device, including determining whether the client device is within a proximity threshold from the casting device. In accordance with a determination that the client device is within the proximity threshold from the casting device, the casting device controls the output device to change from the inactive state to an active state, and receives media content from one or more content sources for display on the output device.

RELATED APPLICATIONS

The present application is related to U.S. patent application Ser. No.______ (Attorney Docket No. 060963-7226-US), filed on Jan. 20, 2016,titled “Methods and Systems for Controlling Media Display in a SmartMedia Display Environment,” which is hereby incorporated by reference inits entirety.

TECHNICAL FIELD

This application relates generally to computer technology, including butnot limited to methods and systems for controlling display of mediacontent on one or more output display devices within a smart mediadisplay environment based on user proximity and/or location detectedwith respect to one or more casting devices.

BACKGROUND

Media display devices have been widely applied to display media contentto audience who gather around the media display devices. As far as theydo not interfere with each other, multiple media display devices can bedistributed at different locations of a media display environment (e.g.,a family household or an office building) to display media contentindividually or simultaneously. Many existing media display devices arepaired with casting devices (e.g., an Apple TV box and a Google TV box)or act as smart television devices to receive television broadcast,satellite broadcast or Internet-based media content. However, theseexisting media display devices have to be powered on and offindividually when a user presses a power button that is located eitheron a media display device or on a remote control device that iscommunicatively coupled to its associated media display device.

Under some circumstances, when a user is moving around in a mediadisplay environment including multiple media display devices (e.g., froma living room to a bedroom in a family household), he or she has topower off a display device, remember a content source that provides thecontent displayed thereon, power on another display device, and tunethis other display device to continue the display of media content thathas been terminated on the previously viewed display device. This couldbe inconvenient for many users who desire to continue to watch the samemedia content while moving around in a media environment.

SUMMARY

Accordingly, there is a need for smart media display control, andspecifically, a need to create a smart media display environment wheredisplay of media content is controlled automatically without userintervention. In various implementations of this application, a smartmedia display environment includes multiple media display devices eachdisposed at a distinct location. Each media display device is coupled toa casting device (e.g., a set top box) that is configured to detect auser's proximity to the casting device. The casting device appliesdetected information of the user proximity to switch the media playerdevice between an active mode and an inactive mode. Furthermore, thecasting devices of the smart media environment determine a user locationwithin the smart media display environment based on the user proximityinformation collected at individual casting devices. The casting devicesthen use the proximity information to support a follow-me mode in whichmedia content that has been watched by a user follows a user to the userlocation and continues to be displayed at a media display device locatedat the detected user location. Such methods optionally complement orreplace conventional methods of requiring an audience to use a remotecontrol or a client device to control the display of the media contentand enabling automatic control of the media display devices without userintervention.

In accordance with one aspect of this application, a method isimplemented on a casting device having one or more processors and memorystoring one or more programs for execution by the one or moreprocessors. The casting device is coupled to an output device, and themethod is implemented to control display of media content on the outputdevice after it is determines that the output device operates at aninactive state. The method includes detecting a client device associatedwith the casting device, and determining a proximity of the clientdevice, including determining whether the client device is within aproximity threshold from the casting device. The method further includesin accordance with a determination that the client device is within theproximity threshold from the casting device, controlling the outputdevice to change from the inactive state to an active state, andreceiving media content from one or more content sources for display onthe output device.

In accordance with another aspect of this application, an electronicdevice is configured to control display of media content on an outputdevice that is coupled to the electronic device. The electronic deviceincludes one or more processors, and memory storing one or more programsto be executed by the one or more processors. The one or more programsfurther include instructions for implementing the operations of theabove method for controlling the display of the media content.

Further, in accordance with one aspect of this application, a method isimplemented on a casting device having one or more processors and memorystoring one or more programs for execution by the one or moreprocessors. The casting device is coupled to an output device, and themethod is implemented to enable a follow-me mode in which the castingdevice controls display of media content on the output device. Themethod includes detecting a client device associated with the castingdevice, and determining a proximity of the client device. The methodfurther includes in accordance with a determination that the clientdevice is within a proximity threshold of the casting device, receivingcontent information associated with content recently accessed by theclient device, obtaining the content from a content source in accordancewith the content information, and outputting the obtained content to anoutput device in accordance with the content information.

In accordance with another aspect of this application, an electronicdevice is configured to control display of media content on an outputdevice that is coupled to the electronic device according to a follow-memode. The electronic device includes one or more processors, and memorystoring one or more programs to be executed by the one or moreprocessors. The one or more programs further include instructions forimplementing the operations of the above method for controlling thedisplay of the media content according to the follow-me mode.

In accordance with some implementations, a casting device includes meansfor performing the operations of any of the methods described above.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the various described implementations,reference should be made to the Description of Implementations below, inconjunction with the following drawings in which like reference numeralsrefer to corresponding parts throughout the figures.

FIG. 1 is an example smart media display environment in accordance withsome implementations.

FIG. 2 is an example operating environment in which casting devicesinteract with a client device or a server system of the smart mediadisplay environment in accordance with some implementations.

FIG. 3 is another example operating environment in which casting devicesinteract with a client device, smart home devices or a server system ofthe smart media display environment in accordance with someimplementations.

FIG. 4 is a block diagram illustrating an example client device that isapplied for automatic control of display of media content in a smartmedia display environment in accordance with some implementations.

FIG. 5 is a block diagram illustrating an example casting device that isapplied for automatic control of display of media content in a smartmedia display environment in accordance with some implementations.

FIG. 6 is a block diagram illustrating an example smart home device in asmart media display environment in accordance with some implementations.

FIG. 7 is a block diagram illustrating an example server system of asmart media display environment in accordance with some implementations.

FIGS. 8A and 8B are two example user interfaces displayed on a displayof a client device for controlling media display associated with mediaapplications in accordance with some implementations.

FIG. 9 is an example user interface displayed on a display of a clientdevice for controlling a schedule for an automatic media output mode anda follow-me mode in accordance with some implementations.

FIGS. 10A and 10B are two example user interfaces displayed on a displayof a client device for controlling an automatic media output mode and afollow-me mode of a casting device in accordance with someimplementations, respectively.

FIG. 11 is another example user interface displayed on a display of aclient device for controlling a schedule for an automatic media outputmode and a follow-me mode of a media application in accordance with someimplementations.

FIG. 12 is an example user interface that displays media content on adisplay of a client device in accordance with some implementations.

FIGS. 13A and 13B are two example user interfaces displayed on a displayof a client device after a casting device detects a proximity of theclient device in accordance with some implementations.

FIGS. 14A and 14B are flow diagrams illustrating a method of controllingdisplay of media content for automatic media output based on userproximity or location in accordance with some implementations.

FIG. 15 is a flow diagram illustrating a method of controlling displayof media content based on user proximity in accordance with someimplementations.

Like reference numerals refer to corresponding parts throughout theseveral views of the drawings.

DESCRIPTION OF IMPLEMENTATIONS

In various implementations of the application, a smart media displayenvironment includes at least one output device that is coupled to acasting device and configured to display media content streamed by thecasting device. The smart display environment is created to control theat least one output device and the display of the media contentautomatically and with no or little user intervention. The castingdevice detects a user's proximity to the casting device, or the user'slocation within the smart media display environment. In someimplementations, the casting device applies detected information of theuser proximity to switch the media player device between an active modeand an inactive mode. For example, the casting device located at aspecific location determines that the user is in proximity, and powerson an output device that is coupled to the casting device. It is notedthat such automatic control of media display does not need userintervention to power on or control the output device when a userapproaches the casting device.

Further, in some implementations, the casting device uses user proximityinformation to support a follow-me mode in which display of mediacontent that has been watched by a user follows a user to a new userlocation and continues to be displayed at an output device located atthe new user location. Specifically, in an example, a user is movingaround in the smart media display environment including multiple outputdevices for displaying media content (e.g., from a living room to abedroom in a family household). When the user leaves the living room,she does not need to power off an output device she has watched orremember a content source that provides the content displayed thereon.Rather, the output device could be automatically turned off or mutedaccording to predetermined media control settings. When the user entersthe bedroom, another output device located there will be powered onautomatically and tuned to continue the display of the media contentfrom where it has been terminated on the previously watched outputdevice. Thus, such methods complement or replace conventional methods ofrequiring an audience to use a remote control or a client device tocontrol the display of the media content, and enables control of themedia display devices automatically and with no or little userintervention.

Reference will now be made in detail to implementations, examples ofwhich are illustrated in the accompanying drawings. In the followingdetailed description, numerous specific details are set forth in orderto provide a thorough understanding of the various describedimplementations. However, it will be apparent to one of ordinary skillin the art that the various described implementations may be practicedwithout these specific details. In other instances, well-known methods,procedures, components, circuits, and networks have not been describedin detail so as not to unnecessarily obscure aspects of theimplementations.

FIG. 1 is an example smart media display environment 100 in accordancewith some implementations. The smart media display environment 100includes a structure 150 (e.g., a house, office building, garage, ormobile home) with various integrated devices. It will be appreciatedthat devices may also be integrated into a smart media displayenvironment 100 that does not include an entire structure 150, such asan apartment, condominium, or office space. The depicted structure 150includes a plurality of rooms 152, separated at least partly from eachother via walls 154. The walls 154 may include interior walls orexterior walls. Each room may further include a floor 156 and a ceiling158.

One or more media device is disposed in the smart media displayenvironment 100 to provide media content that is stored at a localcontent source or streamed from a remote content source. The mediadevices can be classified to two categories: output devices 106 thatdirectly output the media content to audience, and casting devices 108that streams media content to the output devices 108. Examples of theoutput devices 106 include, but are not limited to television (TV)display devices and music players. Examples of the casting devices 108include, but are not limited to, a set-top boxes (STBs), DVD players andTV boxes. In the example smart media display environment 100, the outputdevices 106 are disposed in more than one location, and each outputdevice 106 is coupled to a respective casting device 108 or includes anembedded casting unit. The output device 106-1 includes a TV displaythat is hard wired to a DVD player or a set top box 108-1. The outputdevice 106-2 includes a smart TV device that integrates an embeddedcasting unit to stream media content for display to its audience. Theoutput device 106-3 includes a regular TV display that is coupled to aTV box 108-3 (e.g., Google TV or Apple TV products), and such a TV box108-3 streams media content received from a media content and providesan access to the Internet for displaying Internet-based content on theoutput device 106-3.

In some implementations, an automatic media output mode is implementedat the casting devices 108 located in the smart media displayenvironment 100. In accordance with the automatic media output mode, anoutput device 106 automatically switches to an active state based on aproximity of a client device. Specifically, after a casting device 108determines that an output device 106 coupled thereto operates at aninactive state, the casting device 108 detects a client device 104associated with the casting device 108, and determines that a proximityof the client device 104. The casting device 108 determines whether theclient device 104 is within a proximity threshold from the castingdevice 108. In accordance with a determination that the client device104 is within the proximity threshold from the casting device 108, thecasting device 108 controls the output device to change from theinactive state to an active state, and receives media content from oneor more content sources for display on the output device. It is notedthat when the output device 106 operates at the inactive state, thecasting device 108 could power off the output device 106, ceaseoutputting media content to the output device 106, cause the outputdevice 106 to pause display of media content received from the castingdevice 108 and displayed thereon, cease obtaining media content from anycontent source, cause the output device 106 to reduce an output volumeassociated with media content displayed thereon, or mute the outputdevice 106.

In some implementations, a follow-me mode is implemented at castingdevices 108 located in the smart media display environment 100. Inaccordance with the follow-me mode, display of media content generallyfollows a user when the user moves among different locations in thesmart media display environment 100. Stated another way, when the userreaches a specific location, an output device located at the specificlocation is powered on and tuned to display the media content that waspreviously watched at a different location, allowing the user tocontinue to watch the media content with no or little interruption.Specifically, a first casting device 108 is located at a first locationof the smart media display environment 100, and coupled to a displayinput of a first output device 106 at the first location. The firstcasting device 108 detects a client device 104 associated with the firstcasting device 106. The first casting device 108 determines a proximityof the client device 104. In accordance with a determination that theclient device 104 is within a proximity threshold of the first castingdevice 108, the casting device 108 receives content informationassociated with content recently accessed by the client device 104,obtains the content from a content source in accordance with the contentinformation, and outputs the obtained content to the first output device106 in accordance with the content information.

Further, in some implementations, while outputting the obtained content,the first casting device 106 determines that the client device 104 isoutside the proximity threshold from the first casting device 106. Thefirst client device 106 then implements at least one operation ofpowering off the output device 106, ceasing outputting the obtainedcontent on the output device 108, causing the output display 108 topause the display of the content displayed thereon, ceasing obtainingthe content from the content source in accordance with the contentinformation, causing the output device 108 to reduce an output volumeassociated with the obtained content, and muting the output device. Assuch, in the follow-me mode, when a user moves from a first location toa second location in the smart media display environment, a castingdevice 108 located at the first location optionally implements one ofthe aforementioned operations, while another casting device 108 locatedat the second location detects a proximity of the user and causes itsown output device 106 to display media content that has been displayedto the user at the first location.

In some implementations, smart home devices may be mounted on,integrated with and/or supported by a wall 154, floor 156 or ceiling 158of the smart media display environment 100. The integrated devicesinclude intelligent, multi-sensing, network-connected devices thatintegrate seamlessly with each other in a smart home network and/or witha central server or a cloud-computing system to provide a variety ofuseful smart home functions. In various implementations of theapplication, a smart home device is disposed at the same location of thesmart home environment 100 as a casting device 108 and/or an outputdevice 106, and therefore, is located in proximity to or with a knowndistance with respect to the casting device 108 and the output device106. In some implementations, the smart home device identifies alocation or a proximity of a client device 104, and directly provides tothe casting device information related to the location or the proximityof the client device 104. Alternatively, in some implementations, thesmart home device provides to the casting device 108 the informationrelated to the location or the proximity of the client device 104indirectly via a smart media display server system 140.

It is also noted that in some implementations, a smart home deviceidentifies a location or a proximity of a user in person, and providesthe information related to the location or the proximity of the usereither directly to the casting device 108 or indirectly to the castingdevice 108 via the smart media display server system 140.

The smart home devices in the smart media display environment 100 mayinclude, but are not limited to, one or more intelligent, multi-sensing,network-connected thermostats 122, one or more intelligent,network-connected, multi-sensing hazard detectors 124, one or moreintelligent, multi-sensing, network-connected entryway interface devices126 and 128 (hereinafter referred to as “smart doorbells 126” and “smartdoor locks 128”), one or more intelligent, multi-sensing,network-connected alarm systems 130, one or more intelligent,multi-sensing, network-connected camera systems 132, one or moreintelligent, multi-sensing, network-connected microphone devices 134,and one or more intelligent, multi-sensing, network-connected wallswitches 136. In some implementations, the smart home devices in thesmart media display environment 100 of FIG. 1 includes a plurality ofintelligent, multi-sensing, network-connected appliances 138(hereinafter referred to as “smart appliances 138”), such asrefrigerators, stoves, ovens, televisions, washers, dryers, lights,stereos, intercom systems, garage-door openers, floor fans, ceilingfans, wall air conditioners, pool heaters, irrigation systems, securitysystems, space heaters, window AC units, motorized duct vents, and soforth. Optionally, the smart home devices have a capability of detectinga proximity of a client device or a user.

In some implementations, the smart home devices are configured to detecta presence, proximity and/or location of a user or a client device 104carried by a user. For example, the smart doorbell 126 and/or the smartdoor lock 128 may detect a person's approach to or departure from alocation (e.g., a living room). The smart alarm system 130 may detectthe presence of an individual within close proximity, e.g., usingbuilt-in infrared (IR) sensors. The smart wall plugs 110 may detectoccupancy of a room or enclosure. The camera 132 may capture an image ora video clip at a specific location, and the image or the video clip maybe processed to identify the presence of a user in the specific location(e.g., a kitchen). Moreover, in some situations, specific individualsmay be identified based, for example, on their appearance (e.g., height,face) and/or movement (e.g., their walk/gait) in an image or a videoclip. Alternatively, cameras 132 may additionally include one or moreembedded sensors (e.g., IR sensors and motion detectors) to detect apresence, proximity or location of a user. In addition, a microphonedevice 124 can be regarded as a smart home device in this application,and used to determine occupancy of a location in a smart media displayenvironment. Specifically, audio captured by the microphone device 124may be processed to identify the presence of an occupant in a particularroom. Specific individuals may be identified based, for example, oncharacteristic of their voices.

The smart home devices in the smart media display environment 100 mayadditionally or alternatively include one or more other occupancysensors (e.g., touch screens, IR sensors, ambient light sensors andmotion detectors). In some implementations, the smart home devices inthe smart media display environment 100 include radio-frequencyidentification (RFID) readers (e.g., in each room 152 or a portionthereof) that determine occupancy based on RFID tags located on orembedded in occupants. For example, RFID readers may be integrated intothe smart hazard detectors 104.

In some implementations, in addition to containing sensing capabilities,devices 122, 124, 126, 128, 130, 132, 134, 136 and 138 (which arecollectively referred to as “the smart home devices” or “the smart homedevices 120”) are capable of data communications and information sharingwith other smart home devices, a central server or cloud-computingsystem, and/or other devices (e.g., the client device 104 and thecasting devices 108) that are network-connected. Similarly, the castingdevices 108 are capable of data communications and information sharingwith the smart home devices, a central server or cloud-computing system,and/or other devices (e.g., the client device 104 and the other castingdevices 108) that are network-connected. Data communications may becarried out using any of a variety of custom or standard wirelessprotocols (e.g., IEEE 802.15.4, Wi-Fi, ZigBee, 6LoWPAN, Thread, Z-Wave,Bluetooth Smart, ISA100.11a, WirelessHART, MiWi, etc.) and/or any of avariety of custom or standard wired protocols (e.g., Ethernet, HomePlug,etc.), or any other suitable communication protocol, includingcommunication protocols not yet developed as of the filing date of thisdocument.

In some implementations, the casting devices 108 and the smart homedevices serve as wireless or wired repeaters. In some implementations, afirst one of and the casting devices 108 communicates with a second oneof the casting devices 108 and the smart home devices via a wirelessrouter. The casting devices 108 and the smart home devices may furthercommunicate with each other via a connection (e.g., network interface160) to a network, such as the Internet 112. Through the Internet 112,the casting devices 108 and the smart home devices may communicate witha smart media display server system 140 (also called a central serversystem and/or a cloud-computing system herein). Optionally, the smartmedia display server system 140 may be associated with a manufacturer,support entity, or service provider associated with the casting devices108 and the media content displayed to the user. Accordingly, the smartmedia display server system 140 may include one or more content hosts104 that provide the displayed media content, and a casting device host116 associated with the casting devices 108.

In some implementations, the network interface 160 includes aconventional network device (e.g., a router). The smart media displayenvironment 100 of FIG. 1 further includes a hub device 180 that iscommunicatively coupled to the network(s) 112 directly or via thenetwork interface 160. The hub device 180 is further communicativelycoupled to one or more of the above intelligent, multi-sensing,network-connected devices (e.g., smart home devices of the smart mediadisplay environment 100, the casting devices 108 and the client device104). Each of these network-connected devices optionally communicateswith the hub device 180 using one or more radio communication networksavailable at least in the smart media display environment 100 (e.g.,ZigBee, Z-Wave, Insteon, Bluetooth, Wi-Fi and other radio communicationnetworks). In some implementations, the hub device 180 and devicescoupled with/to the hub device can be controlled and/or interacted withvia an application running on a smart phone, household controller,laptop, tablet computer, game console or similar electronic device. Insome implementations, a user of such controller application can viewstatus of the hub device or coupled network-connected devices, configurethe hub device to interoperate with devices newly introduced to the homenetwork, commission new devices, and adjust or view settings ofconnected devices, etc.

FIG. 2 is an example operating environment in which casting devices 108interact with a client device 104 or a server system 140 of a smartmedia display environment 100 in accordance with some implementations.The smart media display environment 100 includes one or more locations,location A and location B. The user 102 is located at location A or B,and could move freely between these two locations. A casting device 108and an output device 106 are disposed at each of the one or morelocations in the display environment 100, and communicated to each othervia wired or wireless communication available therein. At each location,the respective casting device 108 is configured to obtain media contentor Internet content for display on the output device 106 coupled to thecasting device 108. In various embodiments of this application, thecasting devices 108 are configured to detect a proximity of the user 102or a client device 104 carried by the user 102, and control display ofthe media content displayed on the output devices according to thedetected proximity of the user 102 or the client device 104.

In some implementations, the user 102 is located at a location (e.g.,location A) within the smart media display environment 100, and theoutput device 106-1 operates at an inactive state. At such an inactivestate, the output device 106-1 is optionally powered off, ceasesreceiving media content from the casting device 108-1, pauses display ofany received media content, and/or reduces the volume of its speakerwhile continuing display of media content. On the other hand, thecasting device 108-1 optionally powers off the output device 106-1,ceases outputting media content to the output device 106-1, causes theoutput device 106-1 to pause the display of any received media content,ceases obtaining media content from any content source, causes theoutput device 106-1 to reduce an output volume while continuing displayof media content, and mutes the speaker of the output device 106-1.Alternatively, in some implementations, when the output device 106-1 isa computer-based display device, the inactive mode is one of a power-offmode, a sleep mode and a hibernate mode.

When the output device 106-1 operates at the inactive device, thecasting device 108-1 detects a client device 104 associated with thecasting device 108-1. In some implementations, the client device 104 isdetected, when the casting device 108-1 identifies existence or identityof the client device 104. The casting device 108-1 determines aproximity of the client device 104 including whether the client device104 is within a proximity threshold from the casting device 108-1 (e.g.,within a range of 5 yards). When it is determined that the client device104 is within the proximity threshold from the casting device 108-1, thecasting device 108-1 controls the output device 106-1 to change from theinactive state to an active state, and receives media content from oneor more content sources for display on the output device 106-1. At theactive mode, the output device 106-1 may be awake, fully powered on andready for use. In contrast, in some implementations, when it isdetermined that the client device 104 is outside the proximity thresholdfrom the casting device 108-1, the casting device 108-1 maintains theinactive state at the output device 106-1.

In some implementations, when it enables the change of the output device106-1 from the inactive state to the active state, the casting device104 transmits a prompt to the client device 104. The prompt informs theuser 102 of the change to the active state at the output device 106-1,and provides the user 102 with an option to cancel the change to theactive state. In some implementations, after it enables the change ofthe output device 106-1 from the inactive state to the active state, thecasting device 104 sends a notification to the client device 104 via thecommunication networks 112. The notification informs the user 102 of thechange to the active state at the output device 106-1, while notproviding the user 102 with an option to cancel the change to the activestate. More details of the prompt and the notification displayed on theclient device 104 are discussed below with reference to FIGS. 13A and13B.

In some implementations, when the output device operates at the activestate, the casting device 108-1 determines that the client device 104 isnot within the proximity threshold from the casting device 108-1, andaccordingly, controls the output device 106-1 to change from the activestate to the inactive state. However, in some implementations, thecasting device 108-1 maintains the status quo of the output device106-1, and does not control the output device 106-1 to change from theactive state to the inactive state, when the output device operates atthe active state and the casting device 108-1 determines that the clientdevice 104 is not within the proximity threshold from the casting device108-1 any longer.

In some implementations, the casting device 108-1 detects the clientdevice 104 and/or determines the proximity of the client device 104according to a predefined schedule. Proximity determination could beprohibited at one or more predetermined schedules, e.g., when the use isnot at home or when the user is asleep at night. More details on how toset forth the schedule for device detection and proximity determinationare discussed below with reference to FIG. 9.

The casting device 108-1 could determine the proximity of the clientdevice 104 based on its own detection of the presence of the clientdevice 104. The casting device 108-1 could also determine the proximityof the client device 104 by receiving proximity information from theclient device 104 that has detected the proximity of the casting device108-1. Further, in some implementations, the casting device 108-1provides its own location information to a server 140. The server 140 isconfigured to obtain location information of the client device 104 anddetermine the information of the proximity of the client device based onthe location information of both the casting device 108-1 and the clientdevice 104. The casting device 108-1 then obtains from the server 140information of the proximity of the client device 104. Optionally, theserver 140 is a casting device host 116 that has authenticated theclient device 104. Optionally, the server 140 is a specific content host114 that provides the media content for display on the output device106-1 and recognizes both the casting device 108-1 and the client device104.

Under some circumstances, an active state is enabled for the outputdevice 106-1 when the user is available. The casting device 108-1determines availability of the user 102 in addition to the proximity ofthe user 102 or the client device 104 of the user 102, before thecasting device 108-1 controls the output device 106-1 to change from theinactive state to the active state. In some implementations, the castingdevice 108-1 monitors behavior of the user 102 within a predeterminedperiod of time, and determines the availability of the user 102according to the monitored behavior. In some implementations, thecasting device 108-1 detects that the user 102 is engaged in a telephoneconversation, and that the user 102 is present but not available. Inaccordance with a determination that the user 102 is not available, thecasting device 108-1 deactivates the active state, maintains theinactive state and forgoes a change from the inactive state to theactive state. Alternatively, in some implementations, the casting device108-1 identifies the state of the client device, and determines whetherto activate the output device according to the state of the clientdevice. For example, the casting device 108-1 identifies that the clientdevice is at a mute mode. Then, the casting device 108-1 determines thatthe user is present but not available, and maintains the inactive stateat the output device 106-1.

In some situations, there is another client device 104′ associated withanother user is located at the same location (e.g., location A) wherethe user 102 is currently located. The casting device 108-2 determineswhether it is associated with any other client device (e.g., the clientdevice 104′). When it is determined that the other client device 104′ isalso within the proximity threshold from the casting device 108-1, thecasting device 108-1 further determines that the client device 104 has apriority over the other client device 104′, before it controls theoutput device 106-1 to change from the inactive state to the activestate and receives media content from the one or more content sourcesfor display on the output device 106-1.

In accordance with some implementations of this application, the castingdevices 108 implement a follow-me mode that allows display of mediacontent to follow the user 102 when the user 102 moves in the smartmedia display environment 100 (e.g., moves from location A to location Bor from location B to location A). For example, the user 102 initiallywatches “Hoosiers” on the output device 106-2 at a living room (locationB). When the user 102 leaves the living room and enters his bedroom(location A), the output device 106-1 is automatically powered on, andtuned to continue to play “Hoosiers” from a scene that was last viewedon the output device 106-2 located at the living room. The output device106-2 could be powered off, paused or muted.

At the follow-me mode, the casting device 108-1 detects the clientdevice 104 associated with the casting device 108-1. In someimplementations, the client device 104 is detected when the castingdevice 108-1 identifies existence or identity of the client device 104.The casting device 108-1 determines a proximity of the client device104. When it is determined that the client device is within a proximitythreshold of the casting device 108-1 (e.g., within a range of 5 yards),the casting device 108-1 receives content information associated withcontent recently accessed by the client device 104, and obtains thecontent from a content source in accordance with the contentinformation. Then, the casting device 108-1 outputs the obtained contentto an output device in accordance with the content information. In anexample, the content information includes information of a scene thatwas last viewed in another location (location B) distinct from locationA. The content information is provided to the casting device 108-1 atlocation A, such that the casting device 108-1 could cause the outputdevice 106-1 to play “Hoosters” at location A from the scene that waslast viewed at location B.

It is noted that the content information is not limited to an outputposition of the media content that is last accessed or viewed at adistinct location. In some implementations, the content informationincludes one or more of output position, output quality, output volumeand an identifier of the content source. The obtained content isoutputted to the output device 106-1 from the media content source inaccordance with one or more of the output position, the output qualityand the output volume.

When the user 102 leaves location A, the casting device 108-1 determinesthat the client device 104 is outside the proximity threshold thecasting device 108-1. The casting device 108-1 could update the contentinformation (e.g., output position, output quality, output volume andcontent source information) to the server 140. In some implementations,the client device 104 creates at least a token indicating where thecontent has been stopped at the time of the determination, and updatesthe content information with the token. Likewise, the contentinformation originally provided to the casting device 108-1 waspreviously reported by a casting device located at the other locationdistinct form location A when the user 102 leaves the other location forlocation A.

Assume that the user 102 moves from location A to location B. At thelocation (e.g., location B) to which the user 102 is moved, the castingdevice 108-2 determines that the client device 104 is within a proximitythreshold from the casting device 108-2 and that the casting device108-2 is not associated with any other user or client device in themeantime. The casting device 108-2 receives the content information thathas been updated at a previous location (e.g., location A) by thecasting device 108-1 or the client device 104. The casting device 108-2then obtains the content from the content source in accordance with thecontent information, and outputs the obtained content to the outputdevice 106-1 in accordance with the content information.

In some implementations, the casting devices 108-1 and 108-2 areassociated with a user account of an application (e.g., a casting deviceapplication), and the content information is stored in a memory of aremote server 140 associated with the user account of the application.The casting device 108-2 obtains the content information from the remoteserver 140, after the casting device 108-1 updates the contentinformation stored at the remote server 140. Alternatively, the contentinformation is stored in a local memory of the client device 104, andthe casting device 108-2 receives the content information from theclient device 104.

The client device 104 executes an application for controlling thedisplay of the media content on the output devices 106. Examples of theapplication include a client device application, a casting deviceapplication, a media player application, a TV program application, and asatellite broadcast application. The client device application controlsoperations of the client device 104. The casting device applicationcontrols operations of the casting devices 108. The media playerapplication enables display of media content provided by differentcontent sources (e.g., Netflix, Youtube and Hulu). In someimplementations, the application executed by the client device 104displays a user interface including an affordance, and the affordanceprompts a user to enter a user input to active the follow-me mode. Thedisplayed affordance could be a toggle button. Alternatively, theapplication could also display another affordance on the user interface,and the affordance prompts a user input to activate an automatic mediaoutput mode in which the output devices 106 automatically changes froman inactive state to an active state upon detection of a proximity of aclient device 104. More details on user interfaces for enabling theautomatic media output mode and the follow-me mode are discussed belowwith reference to FIGS. 8A, 8B, 9, 10A, 10B, 11 and 12.

In some situations, there is another client device 104′ associated withanother user and located at the same location as the client device 104.The casting device 108-1 determines whether it is associated with anyother client device (e.g., the client device 104′). When it isdetermined that another client device 104′ is also within the proximitythreshold from the casting device 108-1, the casting device 108-1further determines that the client device 104 has a priority over theother client device 104′, before it obtains the content from a contentsource in accordance with the content information and outputs theobtained content to the output device 106-1 in accordance with thecontent information. Stated another way, the follow-me mode isdeactivated if the client device 104 does not have a priority overanother client device 104′ that exists at the same location and has beenassociated with the casting device 108-1.

Both the automatic media output mode and the follow-mode mode require acasting device 108 to determine a proximity of a client device 104 tothe casting device 108. In some implementations, the proximity of theclient device 104 is determined at the casting device 108 at least byreceiving an audio signal created by a user 102 of the client device 104and determining the proximity of the client device 104 according to theaudio signal. Optionally, the audio signal is triggered by the user 102of the client device inadvertently when the user 102 enters the locationand creates some noise. Optionally, the audio signal is associated witha predetermined voice command, e.g., “Turn on TV,” that has to bespelled out by the user 102 to cause the determination of the proximityof the client device 104.

In some implementations, the casting device 104 further includes anoptical sensor (e.g., an IR sensor). The proximity of the client device104 is determined at the casting device 108 at least by receiving anoptical signal associated with the user of the client device anddetermining the proximity of the client device according to the opticalsignal.

In some implementations, the proximity of the client device 104 isdetermined at the casting device 108 based on a Bluetooth (BT) signalemitted by the client device 104. For example, the casting device 108measures the strength of the BT signal, and determines the proximity ofthe client device 104 based on the strength of the BT signal. In someimplementations, the proximity of the client device 104 is determined atthe casting device 108 via one or more WiFi signals communicated betweenthe client device 104 and the casting device 108 over a local areanetwork. In some implementations, the proximity of the client device 104is determined at the casting device 108 based on a global positioningsystem (GPS) signal associated with the client device. For example, theuser of the client device is approaching a household when the castingdevice determines the proximity of the client device based on the GPSsignal. In some implementations, the proximity of the client device 104is determined at the casting device 108 via two or more of a BT network,a WIFI network and a GPS according to a predetermined operationsequence. In some implementations, the proximity of the client device isdetermined based on at least two of a group consisting of an audiosignal, an optical signal, a BT signal, a WiFi signal, and a GPS signal.

Further, in some implementations, the casting device 108 in the smartmedia display environment 100 may additionally or alternatively includeone or more other occupancy sensors (e.g., touch screens, IR sensors,ambient light sensors and motion detectors). The casting device 108could include radio-frequency identification (RFID) readers thatdetermine occupancy based on RFID tags located in the client device 104.

FIG. 3 is another example operating environment in which casting devices108 interact with a client device 104, smart home devices 120 or aserver system 140 of the smart media display environment 100 inaccordance with some implementations. The smart media displayenvironment 100 includes one or more locations, location A and locationB, and a user 102 is located at location A or B, and could move freelybetween these two locations. A casting device 108, an output device 106and one or more smart home devices 120 are disposed at each of the oneor more locations in the display environment 100, and communicated toeach other via wired or wireless communication available therein. Ateach location, the respective casting device 108 is configured to obtainmedia content or Internet content for display on the output device 106coupled to the casting device 108. Further, at each of the one or morelocations in the display environment 100, at least one of the one ormore smart home devices 120 is disposed at a known spot with respect tothe casting device 108. The at least one of the smart home devices 120is configured to detect a proximity of the user 102 or a client device104 carried by the user 102, and provide information of the detectedproximity to the casting device 108 located at the same location. Thecasting device 108 is configured to control display of the media contentdisplayed on the output devices 106 according to the information of thedetected proximity provided by the at least one of the smart homedevices 120.

As explained above with reference to FIG. 1, the smart home devices 120in the smart media display environment 100 may include, but are notlimited to, smart thermostats 122, smart hazard detectors 124, smartdoorbells 126, smart door locks 128, smart alarm systems 130, camerasystems 132, microphone devices 134, smart wall switches 136, and smartappliances 138. The smart home devices 120 are configured to detect apresence, proximity and/or location of a user or a client device 104carried by a user. In some implementations, a network device (e.g., arouter) could also function as a smart home device 120 for detecting theproximity of the client device 104.

The smart home device 120 could determine the proximity of the clientdevice 104 using an audio sensor or an optical sensor (e.g., a built-inIR sensor). For example, a microphone device 124 can be regarded as asmart home device in this application, and used to determine occupancyof a location in a smart media display environment 100. Specifically,audio captured by the microphone device 124 may be processed to identifythe presence and the proximity of the user 102 in a particular room.Moreover, a camera 132 can also be regarded as a smart home device. Thecamera 132 captures an image or a video clip at a specific location, andthe image or the video clip is processed to identify the presence andproximity of a user 102 in the location (e.g., a kitchen).Alternatively, the cameras 132 may additionally include one or moreembedded sensors (e.g., IR sensors and motion detectors) to detect apresence, proximity or location of a user 102.

The smart home device 120 could also determine the proximity of theclient device 104 based on one or more of a BT signal, a WiFi signal,and a GPS signal associated with the client device 104. Further, in someimplementations, the smart home device 120 in the smart media displayenvironment 100 may additionally or alternatively include one or moreother occupancy sensors (e.g., touch screens, IR sensors, ambient lightsensors and motion detectors). The smart home device 120 could includeradio-frequency identification (RFID) readers that determine occupancybased on RFID tags located in the client device 104.

When a smart home device 120 is disposed at a known spot with respect toa casting device 108, the smart home device 120 or the client device 108is configured to detect a proximity between the smart home device 120and the client device 108, and the casting device 108 obtains from thesmart home device 120 or the client device 108 information concerningthe proximity between the smart home device 120 and the client device108. Such a proximity between the smart home device 120 and the clientdevice 108 is associated with the proximity of the client device 104 tothe casting device 108, because the smart home device 120 is disposed ata known spot with respect to the casting device 108.

Specifically, the smart home device 120 could determine its proximity ofthe client device 104 based on its own detection of the presence of theclient device 104, and provide it to the casting device 108 disposed atthe same location. Alternatively, the client device 104 could determineits proximity of the smart home device 120 based on its own detection ofthe presence of the smart home device 120, and provide it to the castingdevice 108 disposed at the same location. Further, in someimplementations, the smart home device 102 and the client device 104provide their respective location information to a server 140. Theserver is configured to determine an alternative proximity of the clientdevice with respect to the smart home device 102 based on the locationinformation of both the casting device 108-1 and the client device 104.The server 140 could then determine the proximity of the client device104 to the casting device 104 based on the alternative proximity of theclient device 104 and the known spot of the smart home device 102, andnotify the casting device 108 of the proximity of the client device 104.Alternatively, the server could inform the casting device 118 of thealternative proximity of the client device 104, and the casting device108 determines the proximity of the client device based on thealternative proximity of the client device 104 and the known spot of thesmart home device 102. Optionally, the server 140 is a casting devicehost 116 that has authenticated the client device 104. Optionally, theserver 140 is a specific content host 114 that provides the mediacontent for display on the output device 106 and recognizes both thecasting device 108 and the client device 104.

In some implementations, the smart home device 120 is disposed in closeproximity to the casting device 108. The smart home device 120 detect aproximity of the user 102 or a client device 104 carried by the user102. Such a proximity to the smart home device 120 could beapproximately regarded as the proximity of the client device 104 to thecasting device 108 that is located in proximity, and used directly bythe casting device 108 to enable the automatic media output mode or thefollow-me mode.

FIG. 4 is a block diagram illustrating an example client device 104 thatis applied for automatic control of media display in a smart mediadisplay environment 100 in accordance with some implementations.Examples of the client device include, but are not limited to, a mobilephone, a tablet computer and a wearable personal device. The clientdevice 104, typically, includes one or more processing units (CPUs) 402,one or more network interfaces 404, memory 406, and one or morecommunication buses 408 for interconnecting these components (sometimescalled a chipset). The client device 104 includes one or more inputdevices 410 that facilitate user input, such as a keyboard, a mouse, avoice-command input unit or microphone, a touch screen display, atouch-sensitive input pad, a gesture capturing camera, or other inputbuttons or controls. Furthermore, some the client devices 104 use amicrophone and voice recognition or a camera and gesture recognition tosupplement or replace the keyboard. In some implementations, the clientdevice 104 includes one or more cameras, scanners, or photo sensor unitsfor capturing images, for example, of graphic series codes printed onthe electronic devices. The client device 104 also includes one or moreoutput devices 412 that enable presentation of user interfaces anddisplay content, including one or more speakers and/or one or morevisual displays. Optionally, the client device 104 includes a locationdetection device 414, such as a GPS (global positioning satellite) orother geo-location receiver, for determining the location of the clientdevice 104. The client device 104 could also include a proximitydetection device 415, e.g., an IR sensor, for determining a proximity ofa casting device 108 or a smart home device 120.

Memory 406 includes high-speed random access memory, such as DRAM, SRAM,DDR RAM, or other random access solid state memory devices; and,optionally, includes non-volatile memory, such as one or more magneticdisk storage devices, one or more optical disk storage devices, one ormore flash memory devices, or one or more other non-volatile solid statestorage devices. Memory 406, optionally, includes one or more storagedevices remotely located from one or more processing units 402. Memory406, or alternatively the non-volatile memory within memory 406,includes a non-transitory computer readable storage medium. In someimplementations, memory 406, or the non-transitory computer readablestorage medium of memory 406, stores the following programs, modules,and data structures, or a subset or superset thereof:

-   -   Operating system 416 including procedures for handling various        basic system services and for performing hardware dependent        tasks;    -   Network communication module 418 for connecting the client        device 104 to other devices (e.g., the server system 140, the        casting device 108, the smart home devices 120 and the other        client devices 104) via one or more network interfaces 404        (wired or wireless) and one or more networks 112, such as the        Internet, other wide area networks, local area networks,        metropolitan area networks, and so on;    -   User interface module 420 for enabling presentation of        information (e.g., a graphical user interface for presenting        application(s) 426-430, widgets, websites and web pages thereof,        and/or games, audio and/or video content, text, etc.) at the        client device 104 via one or more output devices 412 (e.g.,        displays, speakers, etc.) associated with user interface 410;    -   Input processing module 422 for detecting one or more user        inputs or interactions from one of the one or more input devices        410 and interpreting the detected input or interaction;    -   Web browser module 424 for navigating, requesting (e.g., via        HTTP), and displaying websites and web pages thereof, including        a web interface for logging into a user account associated with        a casting device 108, a media application or a smart home device        120, controlling the casting device 108 or the smart home device        120 if associated with the user account, and editing and        reviewing settings and data that are associated with the user        account;    -   One or more applications for execution by the client device        (e.g., games, social network applications, smart home        applications, and/or other web or non-web based applications for        controlling the casting devices 108 and/04 the smart home        devices 120 and reviewing data captured by such devices),        including one or more of:        -   a casting device application 426 that is executed to provide            client-side functionalities for device provisioning, device            control, and user account management associated with casting            device(s) 108;        -   one or more media player applications 428 that is executed            to provide client-side functionalities for media display and            user account management associated with corresponding media            sources; and        -   one or more smart home device applications 430 that is            executed to provide client-side functionalities for device            provisioning, device control, data processing and data            review of corresponding smart home devices 120; and    -   client data 432 storing at least data associated with automatic        control of media display (e.g., in an automatic media output        mode or a follow-up mode), including:        -   Client device settings 434 for storing information            associated with the client device 104 itself, including            common device settings (e.g., service tier, device model,            storage capacity, processing capabilities, communication            capabilities, etc.), and information for automatic media            display control;        -   Casting device settings 436 for storing information            associated with user accounts of the casting device            application 426, including one or more of account access            information, information for device settings (e.g., service            tier, device model, storage capacity, processing            capabilities, communication capabilities, etc.), and            information for automatic media display control;        -   Media player application settings 438 for storing            information associated with user accounts of one or more            media player applications 428, including one or more of            account access information, user preferences of media            content types, review history data, and information for            automatic media display control;        -   Smart home device settings 440 for storing information            associated with user accounts of the smart home applications            430, including one or more of account access information,            information for smart home device settings (e.g., service            tier, device model, storage capacity, processing            capabilities, communication capabilities, etc.); and        -   Location/proximity data 442 including information associated            with the presence, proximity or location of any of the            client device 104, the smart home devices 120 and the            casting device 108.

In some implementations, each of the casting device application 426, themedia player applications 428 and the smart home device applications 430causes display of a respective user interface on the output device 412of the client device 104. The user interfaces of the applications426-430 could display a notification indicating that change of a displaymode (e.g., the change from the active mode to the inactive mode) ispending or that display of media content is following a user in thesmart media display environment. The user interfaces of the applications426-430 could also prompt the user to enable automatic control of mediadisplay (e.g., in an automatic media output mode or a follow-me mode),create a schedule for such control of media display, set a priorityamong a plurality of users or their client devices, and confirm or forgoa display mode change. More details on the aforementioned userinterfaces are discussed above with reference to FIGS. 8-13.

In some implementations, information for automatic media display controlstored in the client data 432 includes one of more of a first enablesetting for an automatic media output mode, a second enable setting fora follow-me mode, a schedule of automatic media display control, anduser priority information for automatic media display control. Asexplained above, information for automatic media display control couldbe stored in association with one or more of the client device settings434, the casting device settings 436 and the media player applicationsettings 438.

Each of the above identified elements may be stored in one or more ofthe previously mentioned memory devices, and corresponds to a set ofinstructions for performing a function described above. The aboveidentified modules or programs (i.e., sets of instructions) need not beimplemented as separate software programs, procedures, modules or datastructures, and thus various subsets of these modules may be combined orotherwise re-arranged in various implementations. In someimplementations, memory 406, optionally, stores a subset of the modulesand data structures identified above. Furthermore, memory 406,optionally, stores additional modules and data structures not describedabove.

FIG. 5 is a block diagram illustrating an example casting device 108that is applied for automatic control of display of media content in asmart media display environment 100 in accordance with someimplementations. The casting device 108, typically, includes one or moreprocessing units (CPUs) 502, one or more network interfaces 504, memory506, and one or more communication buses 508 for interconnecting thesecomponents (sometimes called a chipset). Optionally, the casting device108 includes a proximity/location detection unit 515, such as an IRsensor, for determining the proximity of a user 102 or a client device104 of the user 102.

Memory 506 includes high-speed random access memory, such as DRAM, SRAM,DDR RAM, or other random access solid state memory devices; and,optionally, includes non-volatile memory, such as one or more magneticdisk storage devices, one or more optical disk storage devices, one ormore flash memory devices, or one or more other non-volatile solid statestorage devices. Memory 506, optionally, includes one or more storagedevices remotely located from one or more processing units 502. Memory506, or alternatively the non-volatile memory within memory 506,includes a non-transitory computer readable storage medium. In someimplementations, memory 506, or the non-transitory computer readablestorage medium of memory 506, stores the following programs, modules,and data structures, or a subset or superset thereof:

-   -   Operating system 516 including procedures for handling various        basic system services and for performing hardware dependent        tasks;    -   Network communication module 518 for connecting the casting        device 108 to other computers or systems (e.g., the server        system 140, the smart home devices 120 and the client device        104) via one or more network interfaces 504 (wired or wireless)        and one or more networks 112, such as the Internet, other wide        area networks, local area networks, metropolitan area networks,        cable television systems, satellite television systems, IPTV        systems, and so on;    -   Content Decoding Module 520 for decoding content signals        received from one or more content sources 114 and outputting the        content in the decoded signals to an output display device 106        coupled to the casting device 108;    -   Proximity/location determination module 522 for determining the        proximity of the client device 104 based on proximity related        information that is detected by the proximity detection unit 515        or provided by the server system 140;    -   Automatic media display module 524 for controlling media        display, e.g., causing automatic media output to the output        device 106 and enabling the display of the output device 106 to        follow a user 102 automatically;    -   casting device data 526 storing at least data associated with        automatic control of media display (e.g., in an automatic media        output mode and a follow-up mode), including:        -   Casting device settings 528 for storing information            associated with user accounts of a casting device            application, including one or more of account access            information, information for device settings (e.g., service            tier, device model, storage capacity, processing            capabilities, communication capabilities, etc.), and            information for automatic media display control;        -   Media player application settings 530 for storing            information associated with user accounts of one or more            media player applications, including one or more of account            access information, user preferences of media content types,            review history data, and information for automatic media            display control; and        -   Location/proximity data 532 including information associated            with the presence, proximity or location of any of the            client device 104, the smart home devices 120 and the            casting device 436.

Information for automatic media display control could be stored inassociation with either the casting device settings 528 or the mediaplayer application settings 530. Information for automatic media displaycontrol includes one of more of a first enable setting for an automaticmedia output mode, a second enable setting for a follow-me mode, aschedule of automatic media display control, and user priorityinformation for automatic media display control.

Each of the above identified elements may be stored in one or more ofthe previously mentioned memory devices, and corresponds to a set ofinstructions for performing a function described above. The aboveidentified modules or programs (i.e., sets of instructions) need not beimplemented as separate software programs, procedures, modules or datastructures, and thus various subsets of these modules may be combined orotherwise re-arranged in various implementations. In someimplementations, memory 506, optionally, stores a subset of the modulesand data structures identified above. Furthermore, memory 506,optionally, stores additional modules and data structures not describedabove.

FIG. 6 is a block diagram illustrating an example smart home device 120in a smart media display environment 100 in accordance with someimplementations. The smart home device 120, typically, includes one ormore processing units (CPUs) 602, one or more network interfaces 604,memory 606, and one or more communication buses 608 for interconnectingthese components (sometimes called a chipset). Optionally, the smarthome device 120 includes a proximity/location detection unit 615, suchas an IR sensor, for determining a proximity of a user 102 or a clientdevice 104 of the user 102. In some implementations, the smart homedevice 120 itself is capable of detecting the presence, location orproximity of a user 102 or a client device 104, and an example of such asmart home device 120 is a camera.

Memory 606 includes high-speed random access memory, such as DRAM, SRAM,DDR RAM, or other random access solid state memory devices; and,optionally, includes non-volatile memory, such as one or more magneticdisk storage devices, one or more optical disk storage devices, one ormore flash memory devices, or one or more other non-volatile solid statestorage devices. Memory 606, optionally, includes one or more storagedevices remotely located from one or more processing units 602. Memory606, or alternatively the non-volatile memory within memory 606,includes a non-transitory computer readable storage medium. In someimplementations, memory 606, or the non-transitory computer readablestorage medium of memory 606, stores the following programs, modules,and data structures, or a subset or superset thereof:

-   -   Operating system 616 including procedures for handling various        basic system services and for performing hardware dependent        tasks for the smart home device 120;    -   Network communication module 618 for connecting the smart home        device 120 to other computers or systems (e.g., the server        system 140, the client device 104, the casting device 108 and        other smart home devices 120) via one or more network interfaces        604 (wired or wireless) and one or more networks 112, such as        the Internet, other wide area networks, local area networks,        metropolitan area networks, and so on;    -   Proximity/location determination module 620 for determining a        location of the smart home device 120 or an alternative        proximity of the client device 104 with respect to the smart        home device 120 that are detected by the proximity detection        unit 615;    -   Smart home device module 622 for enabling the smart home device        120 to implement its designated functions (e.g., for capturing        and generating multimedia data streams and sending the        multimedia data stream to the client device 104 or the server        system 140 as a continuous feed or in short bursts, when the        smart home device 120 includes a video camera);    -   Smart home device data 624 storing at least data associated with        automatic control of media display (e.g., in an automatic media        output mode and a follow-up mode), including location/proximity        data 626 associated with the presence, proximity or location of        any of the client device 104, the smart home devices 120 and the        casting device 436.

Each of the above identified elements may be stored in one or more ofthe previously mentioned memory devices, and corresponds to a set ofinstructions for performing a function described above. The aboveidentified modules or programs (i.e., sets of instructions) need not beimplemented as separate software programs, procedures, modules or datastructures, and thus various subsets of these modules may be combined orotherwise re-arranged in various implementations. In someimplementations, memory 606, optionally, stores a subset of the modulesand data structures identified above. Furthermore, memory 606,optionally, stores additional modules and data structures not describedabove.

FIG. 7 is a block diagram illustrating an example server in the serversystem 140 of a smart media display environment 100 in accordance withsome implementations. An example server is one of a content source host114 and a casting device host 116. The server 140, typically, includesone or more processing units (CPUs) 702, one or more network interfaces704, memory 706, and one or more communication buses 708 forinterconnecting these components (sometimes called a chipset). Theserver 140 could include one or more input devices 710 that facilitateuser input, such as a keyboard, a mouse, a voice-command input unit ormicrophone, a touch screen display, a touch-sensitive input pad, agesture capturing camera, or other input buttons or controls.Furthermore, the server 140 could use a microphone and voice recognitionor a camera and gesture recognition to supplement or replace thekeyboard. In some implementations, the server 140 includes one or morecameras, scanners, or photo sensor units for capturing images, forexample, of graphic series codes printed on the electronic devices. Theserver 140 could also include one or more output devices 712 that enablepresentation of user interfaces and display content, including one ormore speakers and/or one or more visual displays.

Memory 706 includes high-speed random access memory, such as DRAM, SRAM,DDR RAM, or other random access solid state memory devices; and,optionally, includes non-volatile memory, such as one or more magneticdisk storage devices, one or more optical disk storage devices, one ormore flash memory devices, or one or more other non-volatile solid statestorage devices. Memory 706, optionally, includes one or more storagedevices remotely located from one or more processing units 702. Memory706, or alternatively the non-volatile memory within memory 706,includes a non-transitory computer readable storage medium. In someimplementations, memory 706, or the non-transitory computer readablestorage medium of memory 706, stores the following programs, modules,and data structures, or a subset or superset thereof:

-   -   Operating system 716 including procedures for handling various        basic system services and for performing hardware dependent        tasks;    -   Network communication module 718 for connecting the server        system 140 to other devices (e.g., various servers in the server        system 140, the client device 104, the casting device 108, and        the smart home devices 120) via one or more network interfaces        704 (wired or wireless) and one or more networks 112, such as        the Internet, other wide area networks, local area networks,        metropolitan area networks, and so on;    -   User interface module 720 for enabling presentation of        information (e.g., a graphical user interface for presenting        application(s) 426-430, widgets, websites and web pages thereof,        and/or games, audio and/or video content, text, etc.) at the        client device 104;    -   One or more applications for execution on the server side (e.g.,        games, social network applications, smart home applications,        and/or other web or non-web based applications for controlling        the client device 104, the casting devices 108 and the smart        home devices 120 and reviewing data captured by such devices),        including one or more of:        -   a casting device application 722 that is executed to provide            server-side functionalities for device provisioning, device            control, and user account management associated with casting            device(s) 108;        -   one or more media player applications 724 that is executed            to provide server-side functionalities for media display and            user account management associated with corresponding media            sources; and        -   one or more smart home device applications 726 that is            executed to provide server-side functionalities for device            provisioning, device control, data processing and data            review of corresponding smart home devices 120; and    -   Proximity/location determination module 728 for determining the        proximity of the client device 104 to either the casting device        108 or the smart home device 120 based on location information        of any of the client device 104, the smart home device 120 and        the casting device 108;    -   Server system data 730 storing at least data associated with        automatic control of media display (e.g., in an automatic media        output mode and a follow-up mode), including:        -   Client device settings 732 for storing information            associated with the client device 104, including common            device settings (e.g., service tier, device model, storage            capacity, processing capabilities, communication            capabilities, etc.), and information for automatic media            display control;        -   Casting device settings 734 for storing information            associated with user accounts of the casting device            application 722, including one or more of account access            information, information for device settings (e.g., service            tier, device model, storage capacity, processing            capabilities, communication capabilities, etc.), and            information for automatic media display control;        -   Media player application settings 736 for storing            information associated with user accounts of one or more            media player applications 724, including one or more of            account access information, user preferences of media            content types, review history data, and information for            automatic media display control;        -   Smart home device settings 738 for storing information            associated with user accounts of the smart home applications            726, including one or more of account access information,            information for one or more smart home devices 120 (e.g.,            service tier, device model, storage capacity, processing            capabilities, communication capabilities, etc.); and        -   Location/proximity data 740 including information associated            with the presence, proximity or location of any of the            client device 104, the smart home devices 120 and the            casting device 108.

In some implementations, each of the casting device application 722, themedia player applications 724 and the smart home device applications 726causes display of a respective user interface on the output device 712of the client device 104. It is further noted that the server system 140is optionally a server type selected from a casting device host 116, acontent host 114 and a smart home device host. In some implementations,according to its server type, the server system 140 executes one of thecasting device application 722, the media player applications 724 andthe smart home device applications 726 based on corresponding datasettings selected from client device settings 732, casting devicesettings 734, and media player application settings 738.

Further, in some implementations, information for automatic mediadisplay control is optionally stored in the server system data 730 inassociation with one or more of the client device settings 732, thecasting device settings 734 and the media player application settings736. The stored information for automatic media display controlincludes, but is not limited to, one of more of a first enable settingfor an automatic media output mode, a second enable setting for afollow-me mode, a schedule of automatic media display control, and userpriority information for automatic media display control.

Each of the above identified elements may be stored in one or more ofthe previously mentioned memory devices, and corresponds to a set ofinstructions for performing a function described above. The aboveidentified modules or programs (i.e., sets of instructions) need not beimplemented as separate software programs, procedures, modules or datastructures, and thus various subsets of these modules may be combined orotherwise re-arranged in various implementations. In someimplementations, memory 706, optionally, stores a subset of the modulesand data structures identified above. Furthermore, memory 706,optionally, stores additional modules and data structures not describedabove.

FIGS. 8A and 8B are two example user interfaces 800 and 850 displayed ona display of a client device 104 for controlling media displayassociated with media applications in accordance with someimplementations. The user interface 800 is displayed on the display ofthe client device 104, and allows a user to determine the devicesettings of the client device 104 that define characteristics related toautomatic media output for casting device(s) associated with the clientdevice 104. The user interface 800 includes a first affordance 802 forenabling and disabling “Automatic Media Output,” and a first pluralityof media application affordances 804 each representing a mediaapplication available on the client device 104 for controlling displayof media content provided by a corresponding media source. Examples ofthe media applications include, but are not limited to, anInternet-based media application (e.g., Netflix), a radio application,TV applications (e.g., ESPN and Disney), and a satellite TV application.Optionally, the user interface 800 could also be configured to displayan instruction (e.g., “select one of the following media applicationsyou want to play via a casting device automatically”) to prompt a user102 of the client device 104 to select one of the media applications forautomatic media output.

In some implementations, the user 102 enables an automatic media outputmode by selecting the first affordance 802. The user 102 could furtherassociate the automatic media output mode with one or more mediaapplications, when the user 102 selects one or more media applicationsin the list of media applications provided on the user interface 800.For example, automatic media output could be enabled for and limited toTV applications that display ESPN and Disney TV programs. Thus, in someimplementations, when a casting device 108 associated with the clientdevice 104 determines the proximity of the client device 104, thecasting device 108 controls its output device 106 (if operating at aninactive state) to change from the inactive state to an active state,and receives and streams the ESPN or Display TV programs for display onthe output device 106.

The user interface 850 is displayed on the display of the client device104, and allows a user to determine the device settings of the clientdevice 104 that define characteristics related to the follow-me mode forcasting device(s) associated with the client device 104. The userinterface 850 includes a third affordance 806 for enabling and disabling“Follow-Me Mode,” and a second plurality of media applicationaffordances 808 each representing a media application available on theclient device 104 for controlling display of media content provided by acorresponding media source. The media applications associated with thesecond plurality of media application affordances 808 are optionallyidentical to or different from those associated with the first pluralityof media application affordances 804. The user interface 800 could alsodisplay an instruction (e.g., “select one of the following mediaapplications to follow you”) to prompt a user 102 of the client device104 to select one of the media applications for display of correspondingmedia content in the follow-me mode.

In some implementations, the user 102 enables the display of the mediacontent in the follow-me mode by selecting the third affordance 806. Theuser 102 could further associate the follow-me mode with one or moremedia applications, when the user 102 selects one or more mediaapplications in the list of media applications provided on the userinterface 850. For example, the follow-me mode could be enabled for andlimited to TV applications that display ESPN and Disney TV programs,i.e., only TV programs from ESPN and Disney follow the user 102 when theuser 102 moves in the smart home environment 100 including two or morelocations each disposed with a respective casting device 108 and arespective output device 106.

FIG. 9 is an example user interface 900 displayed on a display of aclient device 104 for controlling a schedule for both an automatic mediaoutput mode and a follow-me mode in accordance with someimplementations. The user interface 900 allows a user 102 to determinedevice settings of the client device 104 that define the schedulerelated to the automatic media output mode and the follow-me mode forcasting device(s) 108 associated with the client device 104.Specifically, the user interface 900 includes two affordances 902 and904 for activating “Automatic Media Output” and “Follow-Me Mode,”respectively. The user 102 could enable automatic media output to anoutput device 106 coupled to the client device 104 by selecting theaffordance 902, or enable the display of the media content in thefollow-me mode by selecting the affordance 904.

In addition, the user interface 900 includes a first list of selectableinformation items 906 and a second list of selectable information items908 that are associated with the automatic media output mode and thefollow-me mode, respectively. The first list of selectable informationitems 906 includes one or more schedule information items 906A and anaddition information item 906B. The user 102 could select either theschedule information items 906A or the addition information item 906B.In response to a user selection of one of the schedule information items906A, the selected schedule information item 906A becomes editable andallows the user 102 to modify a schedule for automatic media outputassociated with the selected schedule information item 906A. Further, inresponse to a user selection of the addition information item 906B, apop-up window is displayed on top of the user interface 900, or theaddition information item 906B is expanded within the user interface900, such that the user 102 could enter a new schedule for automaticmedia output of casting device(s) associated with the client device 104.

Similarly, the second list of selectable information items 908 includesone or more schedule information item 908A and an addition informationitem 908B. The user 102 could select either the schedule informationitems 908A or the addition information item 908B. In response to a userselection of one of the schedule information items 908A, the selectedschedule information item 908A becomes editable and allows the user 102to modify a schedule for the follow-me mode associated with the selectedschedule information item 908A. Further, in response to a user selectionof the addition information item 908B, a pop-up window is displayed ontop of the user interface 900, or the addition information item 908B isexpanded within the user interface 900, such that the user 102 couldenter a new schedule for enabling the follow-me mode of castingdevice(s) associated with the client device 104.

In the example shown in FIG. 9, the automatic media output mode isactivated for casting device(s) associated with the client device 104from 7 PM to 11 PM every evening, and the follow-me mode is activatedcasting device(s) associated with the client device 104 from 8 PM to 11PM on the weekdays and from 7 PM to 11:30 PM during the weekend. Statedanother way, the automatic media output mode and the follow-me mode aredeactivated when the user is not at home or when the user is asleep atnight. Thus, a casting device 108 detects the client device 104 anddetermines the proximity of the client device 104 only when the instanttime satisfies the above schedules defined for the automatic mediaoutput mode and the follow-me mode. More details on how to set forth theschedule for device detection and proximity determination are discussedabove with reference to FIG. 2.

FIGS. 10A and 10B are two example user interfaces 1000 and 1050displayed on a display of a client device 104 for controlling anautomatic media output mode and a follow-me mode of a casting device 108in accordance with some implementations, respectively. The userinterface 1000 allows a user to determine settings of a casting deviceapplication for automatic media output associated with one or morecasting devices 108 that have been connected to a user account of thecasting device application. The user interface 1000 includes a firstaffordance 1002 for enabling and disabling “Automatic Media Output.”Optionally, the user interface 1000 further includes a second pluralityof user affordances 1004 each representing a user who is allowed toenable automatic media output. Optionally, the user interface 1000 couldalso be configured to display an instruction or inquiry 1006 (e.g., “Doyou want to output media content automatically when someone (you oranother person) is close to a media output device?” or “Who is allowedfor automatic media output?”) to prompt a user 102 of the client device104 to enable automatic media output for one or more authorized users102.

In some implementations, the user 102 enables an automatic media outputmode by selecting the first affordance 1002. The user 102 could furtherassociate the automatic media output mode with one or more authorizedusers 102, when the user 102 selects one or more users in the list ofusers displayed on the user interface 1000. For example, automatic mediaoutput could be enabled for and limited to a user named “James.” Thus,in some implementations, when a casting device 108 associated with thecasting device application determines a proximity of a client device 104held by James, the casting device 108 controls its output device 106 (ifoperating at an inactive state) to change from the inactive state to anactive state.

Further, the user interface 1050 is displayed on the display of theclient device 104, and allows a user to determine settings of a castingdevice application for the follow-me mode associated with one or morecasting devices 108 that have been connected to a user account of thecasting device application. The user interface 1050 includes a thirdaffordance 1012 for enabling and disabling the “Follow-Me Mode.”Optionally, the user interface 1050 further includes a fourth pluralityof user affordances 1004 each representing a user and the client device104 associated with the respective user that will be followed in thefollow-me mode. Optionally, the user interface 1050 could also beconfigured to display an instruction or inquiry 1006 (e.g., “Do you wantto the displayed media content to follow someone?” or “Who do you wantyour casting device(s) to follow?”) to prompt a user 102 of the clientdevice 104 to enable the follow-me mode for one or more authorized users102.

In some implementations, the user 102 enables the follow-me mode byselecting the third affordance 1012. The user 102 could furtherassociate the follow-me mode with one or more authorized users 102, whenthe user 102 selects one or more users in the list of users displayed onthe user interface 1050. For example, the follow-me mode could beenabled for and limited to a user named “Jill.” Thus, in someimplementations, when a casting device 108 associated with the castingdevice application determines a proximity of a client device 104 held byJill, the casting device 108 controls its output device 106 to continueto display media content that was previously displayed for Jill atanother output device 106 located at a distinct location of the smartmedia display environment 100.

In some implementations not shown in FIGS. 10A and 10B, two or moreusers 102 are selected for the casting device(s) associated with thecasting device application on the user interface 1000 or 1050. The userinterface 1000 or 1050 could be configured to rank the selected two ormore users (e.g., define that a first user 102 has a priority over asecond user 102′). When the casting device 108 determines the proximityto more than one selected user, the casting device 108 furtherdetermines that a first client device 104 of the first user 102 has apriority over the a second client device 104′ of the second user 102′,before it controls its output device 106 to display the media contentthat has been watched by the first user at a distinct location.

FIG. 11 is another example user interface 1100 displayed on a display ofa client device 104 for controlling an automatic media output mode and afollow-me mode of a media application in accordance with someimplementations. The user interface 1100 allows a user to determinesettings of a media application (e.g., Netflix, Amazon Instant, ESPN andDisney) for both an automatic media output mode and a follow-me modeassociated with one or more casting devices 108 that have been coupledto the media application. Specifically, the user interface 1100 includestwo affordances 1102 and 1104 for activating “Automatic Media Output”and “Follow-Me Mode,” respectively. The user 102 could enable anautomatic media output mode for a casting device 108 and its outputdevice 106 by selecting the affordance 1102, or enable the display ofthe media content in the follow-me mode by selecting the affordance1104. Optionally, the user interface 1100 could also display aninstruction or inquiry 1006 (e.g., “Do you want to output media contentautomatically when you are close to a media output device?” or “Do youwant the displayed media content to follow you?”) to prompt a user 102of the client device 104 to enable the automatic media output mode orthe follow-me mode.

In some implementations, the user 102 enables an automatic media outputmode by selecting the first affordance 1102. When a casting device 108detects the client device 104 held by the user 102 and determines aproximity of the client device 104, the casting device 108 controls itsoutput device 106 (if operating at an inactive state) to change from theinactive state to an active state, and display media content provided bya media source associated with the media application. Further, in someimplementations, the user 102 enables the follow-me mode by selectingthe second affordance 1104. Media content provided by a media sourceassociated with the media application is initially streamed by a firstcasting device 108-1 for display on a first output device that iscoupled to the first casting device 108-1. When the user 102 leaves thefirst casting device 108-1 and moves in proximity to a second castingdevice 108-2. The second casting device 108-2 detects the client device104 and determines the proximity of the client device 104. The secondcasting device 108-2 operates under the follow-me mode, controlling asecond output device 106-2 to continue the display of the media contentthat was displayed on the first output device 108-1.

FIG. 12 is an example user interface 1200 that displays media content ona display of a client device 104 in accordance with someimplementations. The media content is configured to be provided by amedia source associate with a media application. The user interface 1200includes a display area 1202 and a plurality of control affordances1204, e.g., a volume control affordance 1204A, a media play affordance1204B, a pause affordance 1204C, a status bar affordance 1204D, anautomatic media output affordance 1204E and a follow-me mode affordance1204F. The affordances 1204A-1204D are applied to control the display ofthe media content associated with the media application (e.g., adjustthe volume, start or pause the display, and select a start frame toreview the display).

In some implementations, the media content is displayed on the displayarea 1202, and the user 102 enables the automatic media output mode byclicking on the affordance 1204E. In response to the click on theaffordance 1204E, the client device 104 generates a request forautomatic media display on an output device 106 based on a proximity ofthe client device 104. In accordance with the automatic media outputmode, a casting device 108 could detect the client device 104 anddetermine the proximity of the client device 104. When the castingdevice 108 determines that the client device 104 is within a proximitythreshold from the casting device 108, the casting device controls theoutput device 106 coupled to the casting device 108 to change from aninactive state to an active state. The casting device 108 then streamsthe media content for display on the output device 106 coupled to thecasting device, while the display area 1202 is either blank or displaysthe same media content. In some implementations, a first click on theaffordance 1204E enables the automatic media output mode, and a secondclick on the affordance 1204E subsequent to the first click disables theautomatic media output mode.

The user 102 could also enable the follow-me mode for the display of themedia content associated with the media application by clicking on theaffordance 1204F. In response to the click on the affordance 1204F, theclient device 104 generates a request for displaying the media contentin the follow-me mode. Casting devices 108 located at differentlocations of a smart media display environment 100 could detect theclient device 104 and determine the proximity of the client device 104.Then, when the user 102 move among the different locations in the smartmedia display environment 100, the casting devices control the outputdevices 106 coupled to the casting devices 108, causing the display ofthe media content on the output devices 106 to track the location of theclient device 104. In some implementations, a first click on theaffordance 1204F enables the follow-me mode for the display of the mediacontent associated with the media application, and a second click on theaffordance 1204F subsequent to the first click disables the follow-memode.

FIG. 13A is an example user interface 1300 displayed on a display of aclient device 104 after a casting device 108 detects a proximity of theclient device 104 in accordance with some implementations. The castingdevice 108 has been enabled for an automatic media output mode. Afterenabling the change of the output device 106 from the inactive state tothe active state, the casting device 104 transmits a prompt to theclient device 104. The prompt informs the user 102 of the change to theactive state at the output device 106, and provides the user 102 with anoption to cancel the change to the active state. For example, a message1302 is displayed on the display of the client device 104, and reads“You are in the proximity range of a TV in your living room. The TV hasbeen powered on and tuned to ESPN. Please disable the media output onthe TV if this is not what you desire.” The user interface 1300 furtherincludes a first affordance 1304 and a second affordance 1306. Inresponse to a respective user click on the first or second affordance,the casting device 108 aborts or finalizes the change of the inactivestate to the active state, respectively. It is also noted that theprompt could also be enabled on the user interface 1300 before theoutput device 106 implements the change from the inactive state to theactive state. More details of the prompt displayed on the client device104 are discussed above with reference to FIG. 2.

FIG. 13B is another example user interface 1350 displayed on a displayof a client device 104 after a casting device 108 detects a proximity ofthe client device 104 in accordance with some implementations. Thecasting device 108 has been enabled for an automatic media output modeor a follow-me mode. After enabling the change of the output device 106from the inactive state to the active state, the casting device 104transmits a notification to the client device 104 displayed on a homescreen of the client device 104. For example, a notification 1312informs the user 102 of the change to the active state at the outputdevice 106 and a TV channel to which the output device 106 is tuned(e.g., ESPN). Another notification 1314 informs the user that thedisplayed media content is currently following the user. If the userwants to change settings for the automatic media output mode or thefollow-me mode, the user needs to unlock the client device 104, andchange the settings in device settings of the client device 104 (FIGS.8A, 8B and 9), a casting device application associated with the castingdevice 108 (FIGS. 10A and 10B), or a media application associated withthe displayed media content (FIGS. 11 and 12).

FIGS. 14A and 14B are flow diagrams illustrating a method 1400 ofcontrolling display of media content for automatic media output based onuser proximity or location in accordance with some implementations. Themethod 1400 is implemented at a casting device (e.g., the casting device108 shown in FIG. 5) having one or more processors and memory storinginstructions for execution by the one or more processors. The castingdevice is coupled to a display input of an output device (e.g., theoutput devices 106-1 and 106-2 shown in FIGS. 2 and 3). Afterdetermining (1402) that the output device operates at an inactive state,the casting device detects (1404) a client device (e.g., the clientdevice 104 shown in FIG. 4) associated with the casting device, anddetermines (1406) a proximity of the client device, includingdetermining whether the client device is within a proximity thresholdfrom the casting device. In accordance with a determination that theclient device is within the proximity threshold from the casting device,the casting device controls (1408) the output device to change from theinactive state to an active state, and receives (1410) media contentfrom one or more content sources for display on the output device.Alternatively, in some implementations, in accordance with adetermination that the client device is outside the proximity thresholdfrom the casting device, the casting device maintains (1412) theinactive state at the output device. In some implementations, inaccordance with the active mode, the output device is awake, fullypowered on, and ready for use, and in accordance with the inactive mode,the output device operates at one of a power-off mode, a sleep mode anda hibernate mode.

In some implementations, when the output device operates at the inactivestate, the casting device implements (1414) one of a group of operationsconsisting of powering off the output device, ceasing outputting mediacontent to the output device, causing the output device to pause displayof media content that is received by the output device, ceasingobtaining media content from any content source, causing the outputdevice to reduce an output volume while continuing display of mediacontent received by the output device, and muting a speaker of theoutput device.

In some implementations, a smart home device is disposed at a known spotwith respect to the casting device, and one of the smart home device andthe client device is configured to detect a proximity between the smarthome device and the client device. The proximity of the client device isdetermined by the casting device by obtaining (1416) from one of a smarthome device and the client device information concerning the proximitybetween the smart home device and the client device. The proximitybetween the smart home device and the client device is associated withthe proximity of the client device to the casting device.

Further, in some implementations, the casting device provides (1418)location information of the casting device to a server (e.g., a serversystem 140), and obtains (1420) from the server information of theproximity of the client device. The server is configured to obtainlocation information of the client device and determine the informationof the proximity of the client device from the location information ofboth the casting device and the client device.

In some implementations, the casting device identifies (1422) a state ofthe client device, and determines (1424) whether to activate the outputdevice according to the state of the client device. For example, thecasting device identifies that the client device is at a mute mode,determines that the user is present but not available, and maintains theinactive state at the output device. In some implementations, thecasting device detects that the user is engaged in a telephoneconversation, and that the user is present but not available. Inaccordance with a determination that the user is not available, thecasting device deactivates the active state, maintains the inactivestate and forgoes a change from the inactive state to the active state.Alternatively, the casting device monitors behavior of the user havingthe client device within a predetermined period of time, and determinesthe availability of the user according to the monitored behavior.

In some implementations, the casting device transmits (1426) a prompt tothe client device. See FIG. 13A. The prompt informs a user of the clientdevice of the change to the active state at the output device andprovides the user with an option to cancel the change to the activestate. Further, in some implementations, the client device executes anapplication for controlling the display of the content on the outputdevice, and the application includes an affordance displayed to receivea user input to activate automatic media output. The applicationexecuted by the client device is one of a casting device application, amedia player application, a TV program application, and a satellitebroadcast application. In an example, the displayed affordance includesa toggle button.

In some implementations, while in the active state, the casting devicedetermines (1428) that the client device is outside the proximitythreshold from the casting device. In accordance with the determinationthat the client device is outside of the proximity threshold from thecasting device, the casting device controls the output device to changefrom the active state to the inactive state.

In some implementations, the casting device detects the client device inaccordance with a predefined schedule. For example, in accordance withthe predefined schedule, proximity determination is prohibited at one ormore predetermined times including a first time duration when the useris not at home, and a second time duration when the user is asleep.

The proximity of the client device could be determined based on an audiosignal, an optical signal, a BT signal, a WiFi signal, a GPS signal orthe like. A proximity/location determination module 522 of the castingdevice 108 is applied to determine the proximity of the client device.More details on proximity determination concerning the client device arediscussed above with reference to at least FIGS. 1 and 2. Alternatively,in some implementations, the proximity of the client device isdetermined based on user intervention. For example, the first castingdevice includes a proximity button, and the proximity of a user carryingthe client device is determined by the casting device when the userpushes the proximity button the casting device.

FIG. 15 is a flow diagram illustrating a method 1500 of controllingdisplay of media content based on user proximity in accordance with someimplementations. The method 1500 is used to implement a follow-me modeat one or more casting devices including a first casting device (e.g.,the casting device 108 shown in FIG. 5) having one or more processorsand memory storing instructions for execution by the one or moreprocessors. The first casting device is coupled to a display input of anoutput device (e.g., the output devices 106-1 and 106-2 shown in FIGS. 2and 3). The first casting device detects (1502) a client deviceassociated with the first casting device, and determines (1504) aproximity of the client device. In accordance with a determination(1506) that the client device is within a proximity threshold of thefirst casting device, the first casting device receives (1508) contentinformation associated with content recently accessed by the clientdevice, obtains (1510) the content from a content source (e.g., thecontent host 114 shown in FIGS. 1-3) in accordance with the contentinformation, and outputs (1512) the obtained content to an output devicein accordance with the content information. In some implementations, theproximity of the client device is determined at a proximity/locationdetermination module 522 of the first casting device, and the output ofthe content is facilitated by an automatic media display module 524.

Further, in some implementations, while outputting the obtained content,in accordance with a determination that the client device is outside theproximity threshold from the first casting device, the casting device108 implements (1514) one operation of a group of operations consistingof: powering off the output device, ceasing outputting the obtainedcontent on the output device, causing the output device to pause thedisplay of the content displayed thereon, ceasing obtaining the contentfrom the content source in accordance with the content information,causing the output device to reduce an output volume associated with theobtained content, and muting a speaker of the output device.

In some implementations, the content information includes one or moreof: output position, output quality, and output volume. The obtainedcontent could be outputted at the output device from the outputposition, according to the output quality and/or at the output volume.In an example, the content information associated with the contentdisplayed on the output device includes information indicating where thedisplay of the content has been ceased at a last review. In anotherexample, in accordance with a determination that the client device isoutside the proximity threshold from the first casting device, the firstcasting device creates a token indicating at least where the content hasbeen stopped at the time of the determination.

In some implementations, the first casting device is located at a firstlocation of a smart media display environment. After the client deviceis outside the proximity threshold from the first casting device, asecond casting device determines that the client device is within aproximity threshold from the second casting device and that the castingdevice is not associated with any other casting device. The secondcasting device receives the content information associated with thecontent, obtains the content from the content source in accordance withthe content information, and outputs the obtained content to the secondoutput device in accordance with the content information. Further, insome situations, the content information includes a token. In accordancewith a determination that the client device is outside the proximitythreshold from the first casting device, the first casting deviceupdates the token indicating at least where the content has been stoppedat the time of the determination. The updated token is received at thesecond casting device, and the obtained content is outputted to thesecond output device according to the token.

In some implementations, the casting device determines whether the firstcasting device is associated with any other client device. Optionally,the casting device determines that the first casting device is notassociated with any other client device when it operates in thefollow-me mode. In an example, the client device includes a first clientdevice. The first casting device determines that a second client deviceis also within the proximity threshold from the casting device andassociated with the first casting device, and that the first clientdevice has a priority over the second client device.

In some implementations, the first casting device is associated with auser account of an application. The content information is stored in amemory of a remote server associated with the user account of theapplication, and the first casting device receives the contentinformation from the remote server. Alternatively, in someimplementations, the content information is stored in a local memory ofthe client device, and the first casting device receives the contentinformation from the client device.

In some implementations, the client device executes an application forcontrolling the display of the content on the output device, and theapplication includes an affordance displayed to receive a user input toactivate the follow-me mode. The application executed by the clientdevice is one of a casting device application, a media playerapplication, a TV program application, and a satellite broadcastapplication. In an example, the displayed affordance includes a togglebutton.

The proximity of the client device could be determined based on an audiosignal, an optical signal, a BT signal, a WiFi signal, a GPS signal orthe like. More details on proximity determination concerning the clientdevice are discussed above with reference to at least FIGS. 1 and 2.Alternatively, in some implementations, the proximity of the clientdevice is determined based on user intervention. For example, the firstcasting device includes a proximity button, and the proximity of a usercarrying the client device is determined by the casting device when theuser pushes the proximity button the casting device.

Methods 1400 and 1500 are, optionally, governed by instructions that arestored in a non-transitory computer readable storage medium and that areexecuted by one or more processors of a casting device (e.g., thecasting device 108). Each of the operations shown in FIGS. 14A, 14B and15 may correspond to instructions stored in the computer memory orcomputer readable storage medium (e.g., memory 506 of the client device108 in FIG. 5). The computer readable storage medium may include amagnetic or optical disk storage device, solid state storage devicessuch as Flash memory, or other non-volatile memory device or devices.The computer readable instructions stored on the computer readablestorage medium may include one or more of: source code, assemblylanguage code, object code, or other instruction format that isinterpreted by one or more processors. Some operations in the method1400 or 1500 may be combined and/or the order of some operations may bechanged.

It should be understood that the particular order in which theoperations in FIGS. 14A and 14B or FIG. 15 have been described aremerely exemplary and are not intended to indicate that the describedorder is the only order in which the operations could be performed. Oneof ordinary skill in the art would recognize various ways to cache anddistribute specific data as described herein. Additionally, it should benoted that details of other processes described with respect to method1400 (e.g., FIGS. 14A and 14B) are also applicable in an analogousmanner to method 1500 described above with respect to FIG. 15, and thatdetails of other processes described with respect to method 1500 (e.g.,FIG. 15) are also applicable in an analogous manner to method 1400described above with respect to FIGS. 14A and 14B. For brevity, thesedetails are not repeated.

It will also be understood that, although the terms first, second, etc.are, in some instances, used herein to describe various elements, theseelements should not be limited by these terms. These terms are only usedto distinguish one element from another. For example, a first type ofaudio feature can be termed a second type of audio feature, and,similarly, a second type of audio feature can be termed a first type ofaudio feature, without departing from the scope of the various describedimplementations. The first type of audio feature and the second type ofaudio feature are both types of audio features, but they are not thesame type of audio feature.

The terminology used in the description of the various describedimplementations herein is for the purpose of describing particularimplementations only and is not intended to be limiting. As used in thedescription of the various described implementations and the appendedclaims, the singular forms “a”, “an” and “the” are intended to includethe plural forms as well, unless the context clearly indicatesotherwise. It will also be understood that the term “and/or” as usedherein refers to and encompasses any and all possible combinations ofone or more of the associated listed items. It will be furtherunderstood that the terms “includes,” “including,” “comprises,” and/or“comprising,” when used in this specification, specify the presence ofstated features, integers, steps, operations, elements, and/orcomponents, but do not preclude the presence or addition of one or moreother features, integers, steps, operations, elements, components,and/or groups thereof.

As used herein, the term “if” is, optionally, construed to mean “when”or “upon” or “in response to determining” or “in response to detecting”or “in accordance with a determination that,” depending on the context.Similarly, the phrase “if it is determined” or “if [a stated conditionor event] is detected” is, optionally, construed to mean “upondetermining” or “in response to determining” or “upon detecting [thestated condition or event]” or “in response to detecting [the statedcondition or event]” or “in accordance with a determination that [astated condition or event] is detected,” depending on the context.

It is to be appreciated that “smart media display environments” mayrefer to smart environments for homes such as a single-family house, butthe scope of the present teachings is not so limited. The presentteachings are also applicable, without limitation, to duplexes,townhomes, multi-unit apartment buildings, hotels, retail stores, officebuildings, industrial buildings, and more generally any living space orwork space.

It is also to be appreciated that while the terms user, customer,installer, homeowner, occupant, guest, tenant, landlord, repair person,and the like may be used to refer to the person or persons acting in thecontext of some particularly situations described herein, thesereferences do not limit the scope of the present teachings with respectto the person or persons who are performing such actions. Thus, forexample, the terms user, customer, purchaser, installer, subscriber, andhomeowner may often refer to the same person in the case of asingle-family residential dwelling, because the head of the household isoften the person who makes the purchasing decision, buys the unit, andinstalls and configures the unit, and is also one of the users of theunit. However, in other scenarios, such as a landlord-tenantenvironment, the customer may be the landlord with respect to purchasingthe unit, the installer may be a local apartment supervisor, a firstuser may be the tenant, and a second user may again be the landlord withrespect to remote control functionality. Importantly, while the identityof the person performing the action may be germane to a particularadvantage provided by one or more of the implementations, such identityshould not be construed in the descriptions that follow as necessarilylimiting the scope of the present teachings to those particularindividuals having those particular identities.

Although various drawings illustrate a number of logical stages in aparticular order, stages that are not order dependent may be reorderedand other stages may be combined or broken out. While some reordering orother groupings are specifically mentioned, others will be obvious tothose of ordinary skill in the art, so the ordering and groupingspresented herein are not an exhaustive list of alternatives. Moreover,it should be recognized that the stages can be implemented in hardware,firmware, software or any combination thereof.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific implementations. However, theillustrative discussions above are not intended to be exhaustive or tolimit the scope of the claims to the precise forms disclosed. Manymodifications and variations are possible in view of the aboveteachings. The implementations were chosen in order to best explain theprinciples underlying the claims and their practical applications, tothereby enable others skilled in the art to best use the implementationswith various modifications as are suited to the particular usescontemplated.

1. A method, comprising: at a casting device having one or moreprocessors and memory storing instructions for execution by the one ormore processors, wherein the casting device is coupled to an outputdevice: determining a state of the output device, wherein the state isan active state or an inactive state; after determining that the outputdevice operates at the inactive state: detecting a client deviceassociated with the casting device; determining a proximity of theclient device, including determining whether the client device is withina proximity threshold from the casting device; in response todetermining that the client device is within the proximity thresholdfrom the casting device: controlling the output device to change fromthe inactive state to the active state, and receiving media content fromone or more content sources for display on the output device.
 2. Themethod of claim 1, wherein when the output device operates at theinactive state, the casting device implements one of a group ofoperations consisting of: powering off the output device; ceasingoutputting media content to the output device; causing the output deviceto pause display of media content that is received by the output device;ceasing obtaining media content from any content source; causing theoutput device to reduce an output volume while continuing display ofmedia content received by the output device; and muting a speaker of theoutput device.
 3. The method of claim 1, further comprising: inaccordance with a determination that the client device is outside theproximity threshold from the casting device, maintaining the inactivestate at the output device.
 4. The method of claim 1, furthercomprising: after determining that the output device is operating at theactive state: determining that the client device is outside theproximity threshold from the casting device; and in accordance with thedetermination that the client device is outside of the proximitythreshold from the casting device, controlling the output device tochange from the active state to the inactive state.
 5. The method ofclaim 1, further comprising: transmitting a prompt to the client device,the prompt informing a user of the client device of the change to theactive state at the output device and providing the user with an optionto cancel the change to the active state.
 6. The method of claim 1,wherein in accordance with the active mode, the output device is awake,fully powered on, and ready for use, and in accordance with the inactivemode, the output device operates at one of a power-off mode, a sleepmode and a hibernate mode.
 7. The method of claim 1, wherein a smarthome device is disposed at a known spot with respect to the castingdevice, and one of the smart home device and the client device isconfigured to detect a proximity between the smart home device and theclient device, further comprises: obtaining from one of the smart homedevice and the client device information concerning the proximitybetween the smart home device and the client device, wherein theproximity between the smart home device and the client device isassociated with the proximity of the client device to the castingdevice.
 8. The method of claim 1, further comprising: providing locationinformation of the casting device to a server; and obtaining from theserver information of the proximity of the client device, wherein theserver is configured to obtain location information of the client deviceand determine the information of the proximity of the client device fromthe location information of both the casting device and the clientdevice.
 9. The method of claim 1, wherein detecting the client devicecomprises detecting the client device in accordance with a predefinedschedule.
 10. The method of claim 9, wherein in accordance with thepredefined schedule, proximity determination is prohibited at one ormore predetermined times including a first time duration when the useris not at home, and a second time duration when the user is asleep. 11.The method of claim 1, wherein the casting device further includes anoptical sensor, and determining the proximity of the client deviceincludes: receiving an optical signal associated with a user of theclient device; determining the proximity of the client device accordingto the optical signal.
 12. The method of claim 1, wherein the proximityof the client device is determined based on at least one of a groupconsisting of an audio signal, an optical signal, a BT signal, a WiFisignal, and a GPS signal.
 13. The method of claim 1, further comprising:monitoring a behavior of a user of the client device within apredetermined period of time; and determining availability of a useraccording to the monitored behavior, wherein the active state is enabledonly when the user is available.
 14. The method of claim 1, furthercomprising: detecting that a user of the client device is engaged in atelephone conversation; determining that the user is present but notavailable; and in accordance with the determination that the user is notavailable, deactivating an active state or maintaining the inactivestate.
 15. The method of claim 1, further comprising: identifying astate of the client device; and determining whether to activate theoutput device according to the state of the client device.
 16. Anelectronic device, wherein the electronic device is coupled to an outputdevice, the electronic device comprising: one or more processors; andmemory storing one or more programs to be executed by the one or moreprocessors, the one or more programs comprising instructions for:determining a state of the output device, wherein the state is an activestate or an inactive state; after determining that the output deviceoperates at the inactive state: detecting a client device associatedwith the electronic device; determining a proximity of the clientdevice, including determining whether the client device is within aproximity threshold from the electronic device; in response todetermining that the client device is within the proximity thresholdfrom the electronic device: controlling the output device to change fromthe inactive state to the active state, and receiving media content fromone or more content sources for display on the output device.
 17. Theelectronic device of claim 16, wherein the client device executes anapplication for controlling the display of the content on the outputdevice, and the application includes an affordance displayed to receivea user input to activate automatic media output.
 18. The electronicdevice of claim 17, wherein the application executed by the clientdevice is one of a casting device application, a media playerapplication, a TV program application, and a satellite broadcastapplication.
 19. A non-transitory computer readable storage mediumstoring one or more programs to be executed by an electronic device,wherein the electronic device is coupled to an output device, the one ormore programs comprising instructions for: determining a state of theoutput device, wherein the state is an active state or an inactivestate; after determining that the output device operates at the inactivestate: detecting a client device associated with the electronic device;determining a proximity of the client device, including determiningwhether the client device is within a proximity threshold from theelectronic device; in response to determining that the client device iswithin the proximity threshold from the electronic device: controllingthe output device to change from the inactive state to the active state,and receiving media content from one or more content sources for displayon the output device.
 20. The non-transitory computer readable storagemedium of claim 19, wherein the client device includes a first clientdevice, further comprising: determining that a second client device isalso within the proximity threshold from the electronic device andassociated with the electronic device, and determining that the firstclient device has a priority over the second client device.
 21. Theelectronic device of claim 16, wherein the one or more programs furthercomprising instructions for: in accordance with a determination that theoutput device is in the active state, maintaining the inactive state atthe output device.
 22. The electronic device of claim 16, wherein theone or more programs further comprising instructions for: afterdetermining that the output device is operating at the active state:determining that the client device is outside the proximity thresholdfrom the casting device; and in accordance with the determination thatthe client device is outside of the proximity threshold from the castingdevice, controlling the output device to change from the active state tothe inactive state.
 23. A system for receiving media content for displayon an output device, said system comprising: at least one casting devicehaving one or more processors and memory storing instructions forexecution by the one or more processors, the at least one casting devicebeing coupled to an output device, wherein the at least one castingdevice is configured to: determine a state of the output device, whereinthe state is an active state or an inactive state; after determiningthat the output device operates at the inactive state: detect a clientdevice associated with the at least one casting device; determine aproximity of the client device, including determining whether the clientdevice is within a proximity threshold from the at least one castingdevice; in response to determining that the client device is within theproximity threshold from the at least one casting device: control theoutput device to change from the inactive state to the active state, andreceive media content from one or more content sources for display onthe output device; and at least one smart home device having one or moreprocessors and memory storing instructions for execution by the one ormore processors, the at least one smart home device being disposed at aknown spot with respect to the at least one casting device, wherein theat least one smart home device is configured to interact with the atleast one casting device.
 24. The system for receiving media content fordisplay on an output device of claim 23, wherein the at least onecasting device is further configured to: in accordance with adetermination that the client device is outside the proximity thresholdfrom the at least one casting device, maintain the inactive state at theoutput device.
 25. The system for receiving media content for display onan output device of claim 23, wherein the at least one casting device isfurther configured to: after determining that the output device isoperating at the active state: determine that the client device isoutside the proximity threshold from the at least one casting device;and in accordance with the determination that the client device isoutside of the proximity threshold from the at least one casting device,control the output device to change from the active state to theinactive state.
 26. The system for receiving media content for displayon an output device of claim 23, wherein the at least one casting deviceis further configured to: transmit a prompt to the client device, theprompt informing a user of the client device of the change to the activestate at the output device and providing the user with an option tocancel the change to the active state.
 27. The system for receivingmedia content for display on an output device of claim 23, wherein: theat least one smart home device is further configured to detect aproximity between the at least one smart home device and the clientdevice; and the casting device is further configured to obtain from theat least one smart home device information concerning the proximitybetween the at least one smart home device and the client device, theproximity between the at least one smart home device and the clientdevice being associated with the proximity of the client device to theat least one casting device.